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ABSTRACT 

Electronic  approach  plate  formats  were  compared  to  determine  which  facilitated  the  best  pilot 
performance  when  flying  precision  and  non-precision  approaches.  Four  formats  which  varied  in  map 
orientation  and  color  scheme  were  flown :  monochrome  north-up,  monochrome  track-up,  color  north-up, 
and  color  track-up.  Although  results  revealed  a  statistically  significant  difference  favoring  the  track-up 
orientation  in  the  non-precision  approaches,  the  differences  were  so  small  that  they  showed  no  practical 
impact  on  performance.  However,  when  given  their  choice  of  which  format  to  fly,  pilots  overwhelmingly 


flew  a  color  map  format.  In  addition,  half  of  the 
track-up. 

INTRODUCTION 

Instrument  approach  procedures  (IAP)  are  designed 
to  provide  a  descent  from  the  enroute  environment  to  a  point 
where  a  safe  landing,  or  if  necessary,  a  missed  approach,  can 
be  made.  Currently,  IAP  information  is  presented  in  paper 
format  and  is  published  and  distributed  by  both  government 
and  commercial  cartographers  as  instrument  approach  charts. 
The  Defense  Mapping  Agency  (DMA)  provides  a  complete 
set  of  military  charts  for  the  United  States  airspace  which 
consists  of  16  bound  booklets,  5  inches  x  8  inches,  with  each 
booklet  containing  approximately  three  hundred  pages  (U.S. 
Department  of  Transportation,  1993).  The  complete  set  of 
military  charts  is  revised  on  an  eight  week  cycle.  The 
commercially  distributed  charts,  primarily  distributed  by 
Jeppesen  Sanderson,  are  revised  on  an  as  required  basis  (U.S. 
Department  of  Defense,  1993).  Depending  on  flight  plan 
requirements,  pilots  may  be  tasked  with  carrying  many  if  not 
all  of  these  books.  Although  pilots  may  only  use  one  of  these 
books  during  the  approach,  storage  of  multiple  books  is 
necessary  in  an  already  crowded  cockpit. 

The  paper  charts,  which  are  black  text  on  a  white 
page,  provide  pilots  with  the  aeronautical  data  required  to 
execute  instrument  approaches  to  airports.  Each  procedure  is 
designated  for  use  with  a  specific  electronic  navigational  aid, 
such  as  an  instrument  landing  system  (ILS),  VHF 
omnidirectional  radio  range  (VOR),  global  positioning  system 
(GPS)  or  tactical  air  navigation  (TACAN). 

The  size  of  the  charts  forces  the  symbology  and  text 
to  be  quite  small  in  order  to  accommodate  all  the  information. 
Fear  of  litigation  and  liability  often  precludes  cartographers 
from  removing  marginally  useful  information  from  the  charts; 
as  a  result,  current  IAPs  tend  to  be  information  dense. 

Electronic  approach  plates  (EAPs)  offer  a  more 
flexible  medium  to  present  approach  information  to  the 
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pilots  flew  a  map  orientation  of  north-up  and  half  flew 

aircrew  (Mykityshyn,  Kuchar,  and  Hansman,  1994). 
Databases  could  be  used  to  provide  the  information  to 
construct  the  format  in  real  time  in  the  cockpit.  Instead  of 
making  revisions  to  a  complete  set  of  manuals,  only  the 
database  would  be  affected  by  changes.  Since  the  database  is 
a  computer  software  item,  it  could  easily  and  inexpensively  be 
updated. 

Due  to  resolution  limitations  of  current  electronic 
displays,  larger  type  fonts  and  symbol  sizes  than  are  currently 
used  on  the  paper  charts  would  be  required  to  improve  symbol 
legibility  (Clay  and  Barlow,  1994).  However,  any  increased 
size  of  the  symbology  results  in  increased  clutter  on 
information  dense  charts.  Decluttering  techniques  or  format 
modifications  will  be  required  for  electronic  charts.  For 
instance,  color  and  the  use  of  a  zooming  capability  could  be 
added  to  the  electronic  displays  to  declutter  the  symbology. 

Another  feature  of  electronic  displays  is  their  ability 
to  provide  dynamic  formats,  while  the  information  on  the 
paper  charts  must  remain  static.  The  approaches  on  the  paper 
charts  are  all  shown  in  a  north-up  orientation,  regardless  of  the 
actual  track  the  pilot  is  flying.  Electronic  displays  can  rotate 
the  symbology,  thus  providing  both  north-up  and  track-up 
formats. 

The  issues  of  color  scheme  and  map  orientation  have 
been  evaluated  in  the  civilian  aircraft  industry  using 
commercial  (Jeppesen)  approach  plates  (Mykityshyn  and 
Hansman,  1991;  Mykityshyn  et  al.,  1994;  Hofer,  Palen, 
Higman,  Infield,  and  Possolo,  1992;  and  Hofer,  Kimball, 
Pepitone,  Higman,  Infield,  and  Possolo,  1993).  Because  the 
military  approach  plates  differ  from  Jeppesen  plates,  DMA 
wanted  to  evaluate  some  of  the  same  issues,  using  their 
military  approach  plates,  to  facilitate  the  transition  from  paper 
approach  plates  to  electronic  formats  in  military  aircraft. 
Therefore,  DMA  asked  Wright  Laboratory  to  evaluate  color 
scheme  and  map  orientation  on  military  approach  plates. 
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OBJECTIVE 


The  purpose  of  this  study  was  to  compare  pilots’ 
performance  using  four  versions  of  EAP  formats.  The 
elements  of  interest  were  color  scheme  and  map  orientation. 
The  four  combinations  tested  were:  monochrome  north-up, 
monochrome  track-up,  color  north-up,  and  color  track-up.  A 
secondary  objective  was  to  investigate  the  use  of  a  zooming 
capability  to  control  the  size  of  the  viewing  area,  thus 
managing  the  amount  of  overall  clutter  on  the  EAP  display 
formats.  A  third  objective  was  to  evaluate  these  formats  in 
both  precision  and  non-precision  approaches  because  of  the 
different  instrument  procedures  used  for  each  type  of 
approach. 


METHOD 


Apparatus 

Dynamic  Cockpit.  This  study  was  conducted  in  a 
fixed-based,  single  seat,  generic  fighter  cockpit  evaluation  tool 
which  contained  a  single  throttle  and  the  limited-displacement 
control  stick  (Figure  1).  An  F-16  aeromodel  was  employed. 


Figure  1.  Cockpit. 

Four  of  the  five  cathode  ray  tubes  (CRTs)  in  the  cockpit  were 
utilized.  The  top  CRT  portrayed  head-up  flight  symbology. 
The  EAPs  were  positioned  on  the  left  6”x8”  portion  of  the 
center  CRT.  The  center  CRT  also  contained  head  down  flight 
symbology  on  the  right.  The  left  CRT  contained  a  crew 
alerting  system  status  display  with  engine  parameters.  The 
bottom  CRT  employed  a  touch  sensitive  keypad  used  to 
change  radio  frequencies,  and  manipulate  commanded 
altitudes  and  airspeed  markers.  Programmable  switches  were 
used  to  change  navigation  radios. 

Display  formats.  Figure  2  shows  an  example  of  an 
EAP  flown  by  the  pilots  during  the  study.  The  amount  of 
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Figure  2.  Electronic  Approach  Plate  Format. 


detail  shown  on  the  EAP  was  less  than  that  on  the  paper 
charts,  but  all  pertinent  information  was  still  present  (Hofer  et 
al.,  1993).  The  plates  were  divided  into  three  sections:  two 
fixed  data  areas,  a  plan-view  diagram,  and  a  vertical  profile 
diagram.  The  fixed  data  areas  contained  numerical  data.  The 
plan-view  was  a  graphical  look-down  view  of  the  entire 
approach.  When  the  plan-view  was  viewed  in  the  track-up 
mode,  the  text  always  remained  upright  as  the  map  rotated 
around  the  fixed  aircraft  symbol  in  the  center  of  the  format. 
The  vertical  profile  view  was  a  side  view  of  the  approach  and 
provided  a  graphical  depiction  primarily  of  altitude 
information. 

In  the  monochrome  version,  all  information  was 
displayed  in  green  on  a  black  background.  In  the  color 
version,  colors  recommended  by  the  Federal  Aviation 
Administration  Advisory  Circular  25-11  (1987)  were  used. 

A  continuous  zoom  function  was  provided  so  that 
subjects  could  change  the  range  of  display  coverage.  This 
allowed  them  to  vary  the  amount  of  detail  displayed  on  both 
the  plan  view  and  the  profile  view  during  different  segments 
of  the  approach.  Pilots  could  change  the  display  range  of 
either  the  plan  view  or  the  profile  view  by  simply  touching  the 
corresponding  section  of  the  display  and  then  using  the  zoom 
switch  to  change  the  range.  The  point  about  which  the  zoom 
focused  in  both  the  plan-view  and  the  profile-view  was  the 
aircraft  symbol. 


Experimental  Design 


This  study  employed  a  mixed  experimental  design. 
The  two  within- subjects  variables  were  color  scheme  (color 
and  monochrome)  and  map  orientation  (north-up  and  track- 
up).  The  between-subjects  variable  was  type  of  approach 
(precision  and  non-precision).  All  variables  were  presented  in 
a  counterbalanced  order. 

Dependent  Variables 

Flight  Performance.  Although  a  number  of 
objective  flight  performance  measures  were  collected,  the 
ones  of  primary  interest  were:  root  mean  square  (RMS) 
airspeed  deviations,  RMS  altitude  deviations,  and  RMS  course 
deviations. 

Subjective  Measures.  Three  types  of  situational 
awareness  (SA)  probe  questions  were  asked  while  the  subjects 
were  flying  the  approaches:  world  referenced,  ego  referenced, 
and  focused  attention  questions  (Hofer  et  al.,  1993).  World 
referenced  questions  asked  location/position  of  two  fixed 
objects,  such  as,  “Is  PARKK  NDB  north  of  SEATTLE 
TACAN?”.  Ego  referenced  questions  asked  location/position 
of  a  object  with  respect  to  the  pilot’s  aircraft,  for  instance,  “Is 
the  airport  to  the  left  of  your  position?”.  Focused  attention 
questions  asked  single  pieces  of  information,  such  as,  “What  is 
the  touchdown  zone  elevation  (TDZE)?”.  Time  to  respond,  as 
well  as  accuracy  of  response  was  recorded. 

Subject-pilots  also  completed  questionnaires  after 
each  profile  was  flown.  After  all  profiles  were  flown,  the 
pilots  filled  out  a  final  questionnaire.  This  questionnaire 
contained  a  Subjective  WORkload  Dominance  (SWORD) 
technique  (Vidulich,  1989),  a  performance  ranking  of  the 
display  formats,  and  the  solicitation  of  overall  comments  on 
the  electronic  approach  formats.  SWORD  enables  a 
subjective  comparison  of  the  difference  in  workload 
experienced  when  flying  the  profiles  using  the  various 
formats. 

Pilots’  Choice.  Previous  research  (Mykityshyn  et  al., 
1994)  comparing  various  electronic  versions  of  commercial 
paper  approach  procedures  (Jeppesen)  has  shown  little 
primary  flight  performance  differences.  Realizing  that 
primary  flight  performance  might  not  be  sensitive  to  the 
versions  of  the  EAPs  compared,  a  key  feature  of  this  study 
was  the  “Pilot’s  Choice”  profile.  This  profile  measured  pilots’ 
preference  of  color  scheme  and  map  orientation  by  allowing 
pilots  to  select  monochrome,  color,  north-up,  and  track-up 
anytime  during  the  flight  of  a  fifth  profile.  Pilots’  Choice  data 
were  gathered  in  two  segments:  from  initial  approach  fix  to 
final  approach  fix,  and  from  final  approach  fix  to  missed 
approach  fix. 

Subjects 


Sessions  lasted  approximately  3  hours.  The  subjects 
were  briefed  first  in  a  classroom  environment.  Then  they  were 
briefed  in  the  cockpit  while  becoming  familiar  with  the 
cockpit  layout  and  the  procedures  for  flying  the  profiles. 
Next,  they  flew  an  approach  using  a  paper  approach  plate. 
This  allowed  subjects  to  become  familiar  with  the  aeromodel 
and  the  SA  probe  questioning  procedures.  Subjects  could 
practice  with  this  approach  more  than  once  if  needed. 

An  approach  plate  with  characteristics  of  the  first 
condition  (the  combination  of  color  scheme  and  map 
orientation)  was  introduced  to  the  subject  statically;  the 
aeromodel  was  not  running,  but  all  graphics  were  present. 
This  briefing  allowed  the  subjects  time  to  review  the  approach 
procedure  and  simulated  what  they  would  have  done  during 
mission  planning  or  while  flying  enroute. 

When  training  on  a  specific  static  format  was 
completed,  pilots  flew  the  approach  for  data  collection.  A 
portion  of  the  questionnaire  was  filled  out  pertaining  to  that 
specific  format.  The  other  formats  were  tested  in  the  same 
fashion.  After  all  four  formats  were  flown,  pilots  flew  the 
Pilot’s  Choice  profile  with  their  choice  of  color  scheme  and 
map  orientation.  The  initial  presentation  of  this  format  was  in 
monochrome  and  north-up,  but  using  the  programmable 
switches,  pilots  could  change  the  color  scheme  and  orientation 
anytime  during  the  approach.  Upon  completion  of  all  flying 
tasks,  pilots  filled  out  the  remainder  of  the  questionnaire. 

RESULTS 

Flight  Performance 

Objective  data  from  this  experiment  were  analyzed 
using  the  Statistical  Package  for  the  Social  Sciences  (SPSS). 
Bartlett’s  test  of  sphericity  was  not  significant  (F(2,14)  = 
31.42,  p  <  0.198)  indicating  independence  of  the  performance 
measures,  therefore,  results  were  analyzed  using  ANOVA. 
Results  showed  a  significant  interaction  between  type  of 
approach  (precision/non-precision)  and  map  orientation 
(north-up/track-up),  in  terms  of  the  RMS  airspeed  deviation 
(F(l,  14)  =  5.48,  p  <  0.035)  (Figure  3).  Analysis  of  simple 
effects  revealed  significantly  less  airspeed  error  when  using 
the  track-up  orientation  than  when  using  the  north-up 
orientation  in  the  non-precision  condition.  No  other 
significant  results  were  found. 

Zoom  Data.  Based  on  a  frequency  count  during 
various  phases  of  the  procedure,  pilots  manipulated  the  zoom 
function  primarily  enroute  to  the  initial  approach  fix  (IAF).  On 
average,  subjects  manipulated  the  zoom  function  a  total  of  12 
times  during  the  “Start  to  IAF”  portion  of  the  flight,  and  1.5 
times  for  all  other  phases  combined.  Figure  4  shows  the 
average  map  range  that  the  pilots  settled  on  for  each  segment 
of  the  procedure. 


Sixteen  Air  Force  pilots  flew  approaches  using  the 
different  formats.  All  were  required  to  have  a  minimum  of 
300  hours  flying  time.  The  subject  pool  consisted  of  6  pilots 
with  primarily  fighter  experience,  8  pilots  with  bomber/cargo 
experience,  and  2  pilots  with  experience  in  both  fighter  and 
transport  aircraft. 


Procedure 
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Figure  3.  Interaction  Between  Type  of  Approach  and  Map 
Orientation. 
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Figure  4. 

Map  Range  by  Flight  Segment 

Subjective  Measures 

SA  Probes.  SA  probe  question  data  was  scored  by 
calculating  the  number  of  correct  responses  versus  the  number 
of  questions  asked.  The  focused  attention  questions  were 
answered  correctly  100%  of  the  time.  The  world  referenced 
questions  were  answered  correctly  95.5%  of  the  time.  In  all 
cases  when  a  world  referenced  question  was  missed,  the  pilots 
were  flying  in  the  track-up  mode  (both  monochrome  and  color 
versions).  For  the  ego  referenced  questions,  correct  answers 
were  given  98%  of  the  time.  The  missed  questions  occurred 
when  the  pilots  were  flying  in  the  monochrome  north-up 
mode. 

SWORD  Data.  The  SWORD  technique  utilized  a 
series  of  pair-wise  comparisons  between  the  various  system 
configurations  to  determine  which  configuration  elicited  the 
most  workload.  The  SWORD  data  were  analyzed  using  a 
2X2X2  Analysis  of  Variance  with  the  two  within  subjects 
factors  as  repeated  measures.  The  ANOVA  results  showed  a 
significant  effect  for  color  scheme  (F(  1 , 14)  =  112.95,  p  < 
0.001).  Figure  5  shows  that  subjective  workload  was  found  to 
be  significantly  higher  with  the  monochrome  display  than  with 
the  color  display. 


Figure  5.  SWORD  Main  Effect  for  Color  Scheme 
Pilot’s  Choice 

Results  for  the  Pilots’  Choice  data  revealed  that  half 
of  the  pilots  flew  north-up,  half  flew  track-  up,  and  all  but  one 
flew  a  color  format. 


DISCUSSION 


Flight  Performance 

Although  analysis  of  RMS  airspeed  error  showed  a 
statistically  significant  interaction  between  the  map  orientation 
variable  and  the  type  of  approach  variable,  there  was  no 
practical  difference  in  airspeed.  For  non-precision 
approaches,  pilots  had  2.38  knots  less  airspeed  error  when 
flying  a  track-up  EAP  than  when  flying  an  EAP  in  the  north- 
up  mode.  Since  pilots  are  normally  required  to  maintain  their 
airspeed  -5  to  +10  knots  during  an  approach  for  a  qualified 
grade  during  an  instrument  evaluation  (US  Air  Force,  1994), 
the  airspeed  deviations  are  well  within  limits. 

Zoom  Data.  Pilots  primarily  used  the  zoom 
capability  in  the  initial  phase  of  the  procedure.  Because  pilots 
are  used  to  seeing  the  entire  approach  at  one  glance  as  on  the 
paper  charts,  pilots  zoomed  the  EAP  until  the  entire  procedure 
was  in  view,  and  then  left  the  zoom  at  that  range  setting  for 
the  entire  procedure.  This  is  confirmed  by  Figure  4,  which 
shows  that  the  average  map  range  for  the  different  segments 
did  not  vary  after  the  initial  adjustment. 

Subjective  Measures 

SA  Probes.  The  SA  probe  data  revealed  that  the 
pilots  had  more  difficulty  answering  world  referenced 
questions  in  the  track-up  mode.  This  is  because  in  the  track- 
up  mode,  the  map  rotates  as  pilots  fly  each  leg  of  the 
approach,  thus,  the  orientation  of  objects  also  changed.  When 
asked  questions  such  as,  “Is  the  parallel  runway  east  of  the 
landing  runway?”,  the  pilots  had  to  mentally  translate  the  map 
information  to  determine  where  north  was  located  and  then 
relate  the  two  objects.  For  the  ego  referenced  questions,  pilots 
had  a  harder  time  while  flying  a  north-up  map.  This  may  be 


due  to  the  fact  that  in  the  north-up  mode,  the  aircraft  moved 
and  at  times  it  could  be  positioned  upside  down  on  the  pilots’ 
map,  in  which  case,  east  was  on  the  pilots'  left  and  west  was 
on  the  pilots’  right.  Therefore,  questions  pertaining  to  the 
location  of  an  object  with  respect  to  the  pilots’  aircraft 
location  were  harder  to  answer. 

SWORD  Data.  The  SWORD  data  showed  that  pilots 
ranked  the  color  formats  as  requiring  less  workload  to 
accomplish  the  task  than  the  monochrome  formats.  In  the 
color  condition,  each  different  set  of  symbols  was  color  coded, 
so  pilots  could  recognize  and  identify  navigation  and 
geographical  features  easier.  In  the  monochrome  condition, 
there  was  much  more  cognitive  workload  involved  in  finding 
and  identifying  objects  and  information. 

Pilots’  Choice 

Color  Scheme  Data.  The  monochrome  format  was 
chosen  by  only  one  pilot  in  the  initial  approach  fix  to  final 
approach  fix  segment,  and  he  changed  to  color  in  the  final 
approach  fix  to  the  missed  approach  point  segment.  All  other 
pilots  chose  color  in  both  segments.  Questionnaire  data 
revealed  that,  even  though  one  pilot  did  fly  a  monochrome 
format,  all  16  preferred  the  color  formats. 

Map  Orientation  Data.  The  pilot’ s  were  evenly  split 
in  their  choice  of  orientation  during  the  segment  from  initial 
approach  fix  to  final  approach  fix  (8  chose  north-up  and  8 
chose  track-up).  During  the  segment  from  final  approach  fix  to 
the  missed  approach  point,  the  results  were  virtually  the  same 
(7  chose  north-up  and  9  chose  track-up).  The  reason  that  some 
of  the  pilots  chose  north-up  could  be  due  to  a  familiarization 
with  the  paper  approach  plates  which  are  always  north-up. 
Conversely,  the  reason  for  choosing  the  track-up  orientation 
could  relate  to  ease  with  which  this  orientation  fits  the  pilots’ 
mental  model  of  where  they  are  in  space.  The  mental  rotation 
involved  in  making  the  north-up  orientation  correspond  to 
their  current  orientation  is  not  needed  in  the  track-up  version. 
To  clarify  this  idea,  think  of  the  physical  rotation  of  a  north-up 
map  involved  in  turning  a  road  map  so  that  it  corresponds  to 
the  direction  of  travel.  Because  the  pilots  cannot  turn  the 
electronic  display,  they  must  perform  the  rotation  mentally. 

IMPLICATIONS  FOR  DESIGNERS 

Based  on  the  results  of  this  study,  color  versions  of 
EAPs  should  be  used.  In  addition,  the  designer  should  provide 
pilot  selectable  north-up  and  track-up  versions  of  the  EAP  -  a 
track-up/north-up  switch  is  an  easy  implementation.  A  zoom 
feature  should  be  included  with  the  center  of  focus  of  the 
zoom 

selectable  by  the  pilot.  In  order  to  reduce  the  need  for 
frequent  adjustments  of  the  zoom  control,  the  initial 
presentation  of  the  procedure  should  be  at  a  scale  sufficient  to 
show  the  entire  approach.  Also,  since  the  zoom  capability 
provides  the  opportunity  to  see  information  at  different  scales. 


there  must  be  upper  and  lower  limits  to  the  size  of  font  and 

symbols  used. 
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